There is a technique for reconstructing a three-dimensional shape of a subject based on image information of the subject photographed by a camera from multiple points of view. For example, the three-dimensional shape of the subject is information in which multiple feature points forming the subject on a three dimension and three-dimensional coordinates are associated.
A conventional technique called structure from motion (SfM) is used to reconstruct the three-dimensional shape from multiple pieces of image information photographed by one camera with a point of view changed FIGS. 29 and 30 are diagrams for explaining the SfM. The SfM executes a procedure 1 and a procedure 2 in order.
The procedure 1 is explained with reference to FIG. 29. The SfM extracts feature points from pieces of image information and matches the feature points most coinciding with one another in the pieces of image information. Pieces of image information 10, 11, and 12 illustrated in FIG. 29 are pieces of image information photographed in different photographing directions and at different timings by one camera. The same subject is included in the pieces of image information 10, 11, and 12.
The SfM calculates feature points of each of the pieces of image information 10 to 12 based on scale-invariant feature transform (SIFT) feature values. In FIG. 29, as an example, feature points 10a, 10b, and 10c are calculated from the image information 10, feature points 11a, 11b, and 11c are calculated from the image information 11, and feature points 12a, 12b, and 12c are calculated from the image information 12. The SfM matches the feature points 10a to 10c, 11a to 11c, and 12a to 12c, respectively. For example, the SfM associates the feature points 10a, 11a, and 12a with each other. The SfM associates the feature points 10b, 11b, and 12b with each other. The SfM associates the feature points 10c, 11c, and 12c with each other.
The procedure 2 is explained with reference to FIG. 30. In the following explanation, a feature point of a subject on a three dimension is referred to as “map point”. A point obtained by projecting the map point on image information based on camera parameters is referred to as “projection point” as appropriate. Projection points 20a to 20c are points obtained by projecting map points 30a to 30c on the image information 10. Projection points 21a to 21c are points obtained by projecting the map points 30a to 30c on the image information 11. Projection points 22a to 22c are points obtained by projecting the map points 30a to 30c on the image information 12.
The SfM associates the feature points and the projection points in each of the pieces of image information and executes a search while changing values of three-dimensional coordinates and camera parameters of the map points such that a square sum of differences between the associated feature points and projection points is minimized. In an example illustrated in FIG. 30, the feature points 10a to 10c are respectively associated with the projection points 20a to 20c. The feature points 11a to 11c are respectively associated with the projection points 22a to 22c. The feature points 12a to 12c are respectively associated with the projection points 22a to 22c. The three-dimensional coordinates of the map points 30a to 30c where a square sum of differences between the associated feature points and projection points is minimized represent a reconstructed three-dimensional shape of the subject.
In the SfM, if noise is included in image information, the three-dimensional coordinates and camera parameters of map points deviating from the optimum values are sometimes estimated because of the noise.
FIG. 31 is a diagram for explaining the influence of noise. In an example illustrated in FIG. 31, a map point 30d occurs as noise. Therefore, in addition to the associating explained with reference to FIG. 30, the associating of a feature point 10d and a projection point 20d, the associating of a feature point 11d and a projection point 21d, and the associating of a feature point 12d and a projection point 22d are performed. In the SfM, values of the three-dimensional coordinates and the camera parameters of the map points are searched out such that the square sum of the differences between the associated feature points and projection points is minimized. Therefore, the search is affected by the noise. In order to solve this problem, the SfM has been executed after removing the noise using a technique called random sample consensus (RANSAC).
Examples of the related art include Japanese Laid-open Patent Publication Nos. 2000-194859, 2002-032745, 2012-208759, and 2014-063376.